Setup¶
Install the Signal Ocean SDK:
pip install signal-ocean
Set your subscription key acquired here: https://apis.signalocean.com/profile
!pip install signal-ocean
signal_ocean_api_key = '' #replace with your subscription key
Call the Voyages API¶
The Voyages API retrieves information about vessel voyages.
from signal_ocean import Connection
from signal_ocean.voyages import VoyagesAPI
from signal_ocean.voyages import Vessel, VesselFilter
from signal_ocean.voyages import VesselType, VesselTypeFilter
from signal_ocean.voyages import VesselClass, VesselClassFilter
import pandas as pd
from datetime import date, timedelta, datetime
import plotly.express as px
connection = Connection(signal_ocean_api_key)
api = VoyagesAPI(connection)
Get vessel class id for VLCCsTankers
vessel_classes = api.get_vessel_classes()
vessel_classes_df = pd.DataFrame(v.__dict__ for v in vessel_classes)
vessel_classes_df[vessel_classes_df['vessel_type']=='Tanker'].head(7)
| vessel_class_id | vessel_class_name | vessel_type_id | vessel_type | |
|---|---|---|---|---|
| 11 | 84 | VLCC | 1 | Tanker |
| 12 | 85 | Suezmax | 1 | Tanker |
| 13 | 86 | Aframax | 1 | Tanker |
| 14 | 87 | Panamax | 1 | Tanker |
| 15 | 88 | MR2 | 1 | Tanker |
| 16 | 89 | MR1 | 1 | Tanker |
| 17 | 90 | Small | 1 | Tanker |
Get voyages for VLCCs¶
For VLCCs we will use (vessel_class_id = 84) when calling the API
voyages = api.get_voyages_condensed(vessel_class_id=84, date_from=datetime.strptime("2019-09-01", "%Y-%m-%d"))
voyages = pd.DataFrame([v.__dict__ for v in voyages])
voyages.tail()
| imo | voyage_number | vessel_type_id | vessel_class_id | vessel_status_id | commercial_operator_id | deleted | events | id | horizon_id | ... | last_discharge_sailing_date | last_discharge_country_id | last_discharge_country_name | last_discharge_area_id_level0 | last_discharge_area_name_level0 | repairs_ind | storage_ind | sts_load_ind | sts_discharge_ind | local_trade_ind | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 16257 | 9941673 | 4 | 1 | 84 | 1 | 2236.0 | False | None | I97B2A954VEDBD3C900 | 1 | ... | 2023-08-31 23:55:54+00:00 | 129.0 | Korea, Republic of | None | Korea | True | False | False | False | False |
| 16258 | 9941673 | 5 | 1 | 84 | 1 | 2236.0 | False | None | I97B2A954VEDC71FD00 | 2 | ... | NaT | NaN | None | None | None | False | False | False | False | True |
| 16259 | 9943748 | 1 | 1 | 84 | 1 | 1645.0 | False | None | I97BAC454VEDBD3C900 | 1 | ... | 2023-08-19 19:57:33+00:00 | 173.0 | Netherlands | None | Continent | True | False | True | False | False |
| 16260 | 9943748 | 2 | 1 | 84 | 1 | 1645.0 | False | None | I97BAC454VEDC71FD00 | 2 | ... | 2023-11-02 02:36:13.823000+00:00 | 112.0 | India | None | Pakistan / West Coast India | False | False | False | False | False |
| 16261 | 9946673 | 1 | 1 | 84 | 1 | NaN | False | None | I97C63154VEDB84AF00 | 2 | ... | NaT | NaN | None | None | None | False | False | False | False | True |
5 rows × 83 columns
Convert Timestamp to datetime, drop voyages with first_load_sailing_date in the future (ongoing or future)
voyages['first_load_sailing_date'] = voyages['first_load_sailing_date'].apply(lambda d : pd.to_datetime(d).tz_localize(None))
voyages=voyages[voyages['first_load_sailing_date']<datetime.now()]
Examine US VLCC Dirty Exports¶
Group by origin and destination (first load, last discharge), and use first_load_sailing_date as date of export
dirty_exports_country_level = voyages[(voyages['first_load_sailing_date']>"2020-01-01")&(voyages['cargo_group']=='Dirty')] \
.set_index('first_load_sailing_date') \
.groupby(['first_load_country_name', 'last_discharge_country_name'])['quantity'] \
.resample('MS') \
.sum() \
.reset_index() \
.rename(columns={'first_load_sailing_date': 'Date'})
dirty_exports_country_level.head()
| first_load_country_name | last_discharge_country_name | Date | quantity | |
|---|---|---|---|---|
| 0 | Algeria | Netherlands | 2020-05-01 | 257000.0 |
| 1 | Angola | Angola | 2020-02-01 | 246000.0 |
| 2 | Angola | Benin | 2023-04-01 | 192000.0 |
| 3 | Angola | Brazil | 2022-11-01 | 230000.0 |
| 4 | Angola | Caribs | 2021-10-01 | 225000.0 |
Drop intra country flows
dirty_exports_country_level = dirty_exports_country_level[dirty_exports_country_level['first_load_country_name']!=dirty_exports_country_level['last_discharge_country_name']]
- Keep only US exports
- Aggregate all exports monthly
- Drop unnecessary columns
us_dirty_exports = dirty_exports_country_level[(dirty_exports_country_level['first_load_country_name']=='United States')].copy()
us_dirty_exports['TotalExports'] = us_dirty_exports['quantity'].groupby(us_dirty_exports['Date']).transform('sum')
us_dirty_exports.drop_duplicates(subset=['Date'], keep='last', inplace = True)
us_dirty_exports.drop(['last_discharge_country_name', 'quantity'], axis=1, inplace = True)
us_dirty_exports.head()
| first_load_country_name | Date | TotalExports | |
|---|---|---|---|
| 8655 | United States | 2023-08-01 | 8971000.0 |
| 8656 | United States | 2023-09-01 | 1297000.0 |
| 8657 | United States | 2020-01-01 | 4319000.0 |
| 8658 | United States | 2020-02-01 | 4207000.0 |
| 8701 | United States | 2020-03-01 | 5032000.0 |
Plot US VLCC Dirty exports¶
us_dirty_exports['Year'] = us_dirty_exports['Date'].apply(lambda date : date.year)
us_dirty_exports['Month'] = us_dirty_exports['Date'].apply(lambda date : date.month)
us_dirty_exports.sort_values(['Month', 'Year'], inplace = True)
us_dirty_exports['Month'] = us_dirty_exports['Date'].apply(lambda date : date.strftime("%b"))
fig = px.histogram(us_dirty_exports, x="Month", y="TotalExports", color='Year',
barmode='group', height=500, width=1000,
title='US VLCC Dirty Exports (MT)',
color_discrete_sequence=["lightgray", "gray", "lightblue"])
fig.show()
Examine China VLCC Dirty Imports
Convert Timestamp to datetime, drop uncompleted voyages (with last_discharge_arrival_date in the future)voyages['last_discharge_arrival_date'] = voyages['last_discharge_arrival_date'].apply(lambda d : pd.to_datetime(d).tz_localize(None))
voyages=voyages[voyages['last_discharge_arrival_date']<datetime.now()]
Group by origin and destination (first load, last discharge), and use last_discharge_arrival_date as date of import
dirty_imports_country_level = voyages[(voyages['last_discharge_arrival_date']>"2020-01-01")&(voyages['cargo_group']=='Dirty')] \
.set_index('last_discharge_arrival_date') \
.groupby(['first_load_country_name', 'last_discharge_country_name'])['quantity'] \
.resample('MS') \
.sum() \
.reset_index() \
.rename(columns={'last_discharge_arrival_date': 'Date'})
Drop intra country flows
dirty_imports_country_level = dirty_imports_country_level[dirty_imports_country_level['first_load_country_name']!=dirty_imports_country_level['last_discharge_country_name']]
dirty_imports_country_level.head()
| first_load_country_name | last_discharge_country_name | Date | quantity | |
|---|---|---|---|---|
| 0 | Algeria | Netherlands | 2020-05-01 | 257000.0 |
| 2 | Angola | Benin | 2023-05-01 | 192000.0 |
| 3 | Angola | Brazil | 2022-12-01 | 230000.0 |
| 4 | Angola | Caribs | 2021-10-01 | 225000.0 |
| 5 | Angola | China | 2020-01-01 | 3131000.0 |
- Keep only China imports
- Aggregate all exports monthly
- Drop unnecessary columns
china_dirty_imports = dirty_imports_country_level[(dirty_imports_country_level['last_discharge_country_name']=='China')].copy()
china_dirty_imports['TotalImports'] = china_dirty_imports['quantity'].groupby(china_dirty_imports['Date']).transform('sum')
china_dirty_imports.drop_duplicates(subset=['Date'], keep='last', inplace = True)
china_dirty_imports.drop(['first_load_country_name', 'quantity'], axis=1, inplace = True)
china_dirty_imports.head()
| last_discharge_country_name | Date | TotalImports | |
|---|---|---|---|
| 7462 | China | 2023-09-01 | 7136000.0 |
| 7991 | China | 2020-01-01 | 29863000.0 |
| 8759 | China | 2023-08-01 | 38645000.0 |
| 8828 | China | 2023-06-01 | 39817000.0 |
| 8829 | China | 2023-07-01 | 35074000.0 |
Plot China VLCC Dirty Imports¶
china_dirty_imports['Year'] = china_dirty_imports['Date'].apply(lambda date : date.year)
china_dirty_imports['Month'] = china_dirty_imports['Date'].apply(lambda date : date.month)
china_dirty_imports.sort_values(['Month', 'Year'], inplace = True)
china_dirty_imports['Month'] = china_dirty_imports['Date'].apply(lambda date : date.strftime("%b"))
fig = px.histogram(china_dirty_imports, x="Month", y="TotalImports", color='Year',
barmode='group', height=500, width=1000,
title='China VLCC dirty Imports',
color_discrete_sequence=["lightgray", "gray", "lightblue"])
fig.show()
Build your custom Flows - US Clean Exports to specific countries by Panamax and MR2¶
Get vessel class ids for Tanker vessel classes
vessel_classes = api.get_vessel_classes()
vessel_classes_df = pd.DataFrame(v.__dict__ for v in vessel_classes)
vessel_classes_df[vessel_classes_df['vessel_type']=='Tanker'].head(10)
| vessel_class_id | vessel_class_name | vessel_type_id | vessel_type | |
|---|---|---|---|---|
| 11 | 84 | VLCC | 1 | Tanker |
| 12 | 85 | Suezmax | 1 | Tanker |
| 13 | 86 | Aframax | 1 | Tanker |
| 14 | 87 | Panamax | 1 | Tanker |
| 15 | 88 | MR2 | 1 | Tanker |
| 16 | 89 | MR1 | 1 | Tanker |
| 17 | 90 | Small | 1 | Tanker |
Select the vessel classes of interest¶
Get voyages for selected vessel classes (Panamax and MR2) after a given date
vessel_class_ids = (87, 88) # Get only voyages for Aframax, Panamax and MR2
date_from = datetime.strptime("2021-09-01", "%Y-%m-%d")
voyages = []
for vessel_class_id in vessel_class_ids:
voyages.append(api.get_voyages_condensed(vessel_class_id=vessel_class_id, date_from=date_from))
voyages = pd.DataFrame([v.__dict__ for vc in voyages for v in vc])
voyages.tail(5)
| imo | voyage_number | vessel_type_id | vessel_class_id | vessel_status_id | commercial_operator_id | deleted | events | id | horizon_id | ... | last_discharge_sailing_date | last_discharge_country_id | last_discharge_country_name | last_discharge_area_id_level0 | last_discharge_area_name_level0 | repairs_ind | storage_ind | sts_load_ind | sts_discharge_ind | local_trade_ind | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 50807 | 9960306 | 1 | 1 | 88 | 1 | 1216.0 | False | None | I97FB7258VEDBAC3C00 | 1 | ... | 2023-06-23 07:57:39+00:00 | 232.0 | Turkey | None | East Mediterranean | True | False | False | False | False |
| 50808 | 9960306 | 2 | 1 | 88 | 1 | 1216.0 | False | None | I97FB7258VEDC22E300 | 1 | ... | 2023-08-13 03:56:05+00:00 | 205.0 | Singapore | None | Singapore / Malaysia | False | False | False | False | False |
| 50809 | 9960306 | 3 | 1 | 88 | 1 | 1216.0 | False | None | I97FB7258VEDC4A7000 | 2 | ... | 2023-09-07 22:40:38.845000+00:00 | 205.0 | Singapore | None | Singapore / Malaysia | False | False | False | False | False |
| 50810 | 9967249 | 1 | 1 | 88 | 1 | NaN | False | None | I98169158VEDC4A7000 | 2 | ... | NaT | NaN | None | None | None | True | False | False | False | True |
| 50811 | 9967251 | 1 | 1 | 88 | 1 | NaN | False | None | I98169358VEDC4A7000 | 2 | ... | NaT | NaN | None | None | None | True | False | False | False | True |
5 rows × 83 columns
Date of flows will be based on the export date
- Convert Timestamp to datetime, drop voyages with first_load_sailing_date in the future (ongoing or future)
voyages['first_load_sailing_date'] = voyages['first_load_sailing_date'].apply(lambda d : pd.to_datetime(d).tz_localize(None))
voyages=voyages[voyages['first_load_sailing_date']<datetime.now()]
voyages.head()
| imo | voyage_number | vessel_type_id | vessel_class_id | vessel_status_id | commercial_operator_id | deleted | events | id | horizon_id | ... | last_discharge_sailing_date | last_discharge_country_id | last_discharge_country_name | last_discharge_area_id_level0 | last_discharge_area_name_level0 | repairs_ind | storage_ind | sts_load_ind | sts_discharge_ind | local_trade_ind | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 9063108 | 191 | 1 | 87 | 1 | 356.0 | False | None | I8A4AC4VED8D72300 | 1 | ... | 2021-10-28 03:57:47+00:00 | 55.0 | China | None | South China | False | False | False | False | True |
| 2 | 9063108 | 192 | 1 | 87 | 1 | 356.0 | False | None | I8A4AC4VED90BDF00 | 1 | ... | 2021-12-03 07:58:12+00:00 | 55.0 | China | None | South China | True | False | False | False | True |
| 3 | 9063108 | 193 | 1 | 87 | 1 | 356.0 | False | None | I8A4AC4VED9336C00 | 1 | ... | 2021-12-19 23:54:26+00:00 | 55.0 | China | None | South China | False | False | False | False | True |
| 4 | 9063108 | 194 | 1 | 87 | 1 | 356.0 | False | None | I8A4AC4VED94DCA00 | 1 | ... | 2022-01-07 07:53:53+00:00 | 55.0 | China | None | South China | False | False | False | False | True |
| 5 | 9063108 | 195 | 1 | 87 | 1 | 356.0 | False | None | I8A4AC4VED95AF900 | 1 | ... | 2022-01-22 23:58:57+00:00 | 55.0 | China | None | South China | False | False | False | False | True |
5 rows × 83 columns
Select the cargo types of interest¶
Examine different cargo types under the "Clean" cargo group
voyages[voyages['cargo_group']=='Clean'].drop_duplicates(subset=['cargo_group', 'cargo_type'], keep='last')[['cargo_group', 'cargo_type']].head()
| cargo_group | cargo_type | |
|---|---|---|
| 18548 | Clean | Gasoline 92 MON |
| 19287 | Clean | C9 |
| 21443 | Clean | CBOB |
| 22052 | Clean | C6 |
| 22419 | Clean | Low Sulphur Gasoil |
voyages[(voyages['first_load_country_name']=='United States')&(voyages['cargo_group']=='Clean')]['cargo_type'].value_counts().head(10)
Gasoline 2578 Diesel 1765 Ultra Low Sulphur Diesel 945 Gasoil 498 Unleaded Motor Spirit 435 Naphtha 424 Jet 407 Gasoline Blending Component 103 Gasoline 92 RON 45 LSD 36 Name: cargo_type, dtype: int64
- Keep cargo types of interest
- Group by origin and destination (first load, last discharge) and cargo type
- Use last_discharge_arrival_date as date of flow
cargo_types_of_interest = ('Gasoline', 'Diesel', 'Ultra Low Sulphur Diesel', 'Gasoil', 'Naphtha', 'Unleaded Motor Spirit', 'Jet')
custom_flows_country_level = voyages[(voyages['first_load_sailing_date']>"2022-01-01")&(voyages['cargo_type'].isin(cargo_types_of_interest))] \
.set_index('first_load_sailing_date') \
.groupby(['first_load_country_name', 'last_discharge_country_name', 'cargo_type'])['quantity'] \
.resample('MS') \
.sum() \
.reset_index() \
.rename(columns={'first_load_sailing_date': 'Date'})
custom_flows_country_level.head()
| first_load_country_name | last_discharge_country_name | cargo_type | Date | quantity | |
|---|---|---|---|---|---|
| 0 | Albania | Libya | Gasoline | 2022-05-01 | 31000.0 |
| 1 | Albania | Slovenia | Ultra Low Sulphur Diesel | 2023-08-01 | 30000.0 |
| 2 | Algeria | Belgium | Jet | 2023-02-01 | 37000.0 |
| 3 | Algeria | Belgium | Naphtha | 2022-01-01 | 30000.0 |
| 4 | Algeria | Belgium | Naphtha | 2022-02-01 | 37000.0 |
Select flows of interest¶
Keep only flows between US and (Mexico, Brazil, Argentina)
import_countries = ('Mexico', 'Brazil', 'Argentina')
custom_flows = custom_flows_country_level[(custom_flows_country_level['first_load_country_name']=='United States')&(custom_flows_country_level['last_discharge_country_name'].isin(import_countries))].copy()
custom_flows.head()
| first_load_country_name | last_discharge_country_name | cargo_type | Date | quantity | |
|---|---|---|---|---|---|
| 21550 | United States | Argentina | Diesel | 2022-02-01 | 130000.0 |
| 21551 | United States | Argentina | Diesel | 2022-03-01 | 0.0 |
| 21552 | United States | Argentina | Diesel | 2022-04-01 | 72000.0 |
| 21553 | United States | Argentina | Diesel | 2022-05-01 | 170000.0 |
| 21554 | United States | Argentina | Diesel | 2022-06-01 | 34000.0 |
Plot custom flows - US Clean Exports to specific countries¶
custom_flows['Year'] = custom_flows['Date'].apply(lambda date : date.year)
custom_flows['Month'] = custom_flows['Date'].apply(lambda date : date.month)
custom_flows.sort_values(['Year', 'Month'], inplace = True)
custom_flows['Month'] = custom_flows['Date'].apply(lambda date : date.strftime("%b"))
custom_flows['Year'] = custom_flows['Date'].apply(lambda date : date.strftime("%Y"))
custom_flows['MonthYear'] = custom_flows['Month'].astype(str) + ' ' + custom_flows['Year'].astype(str)
fig = px.histogram(custom_flows, x="MonthYear", y="quantity", color='cargo_type',
height=500, width=1000, title='custom Flows - US Clean Exports to (Mexico, Brazil and Argentina)',
color_discrete_sequence=px.colors.qualitative.Pastel)
fig.update_layout(bargap=0.2)
fig.show()